MTurk Crowdsourcing: A Viable Method for Rapid Discovery of Arabic Nicknames?
نویسندگان
چکیده
This paper presents findings on using crowdsourcing via Amazon Mechanical Turk (MTurk) to obtain Arabic nicknames as a contribution to exiting Named Entity (NE) lexicons. It demonstrates a strategy for increasing MTurk participation from Arab countries. The researchers validate the nicknames using experts, MTurk workers, and Google search and then compare them against the Database of Arabic Names (DAN). Additionally, the experiment looks at the effect of pay rate on speed of nickname collection and documents an advertising effect where MTurk workers respond to existing work batches, called Human Intelligence Tasks (HITs), more quickly once similar higher paying HITs are posted.
منابع مشابه
Assessing Pictograph Recognition: A Comparison of Crowdsourcing and Traditional Survey Approaches
BACKGROUND Compared to traditional methods of participant recruitment, online crowdsourcing platforms provide a fast and low-cost alternative. Amazon Mechanical Turk (MTurk) is a large and well-known crowdsourcing service. It has developed into the leading platform for crowdsourcing recruitment. OBJECTIVE To explore the application of online crowdsourcing for health informatics research, spec...
متن کاملCrowdsourcing Music Similarity Judgments using Mechanical Turk
Collecting human judgments for music similarity evaluation has always been a difficult and time consuming task. This paper explores the viability of Amazon Mechanical Turk (MTurk) for collecting human judgments for audio music similarity evaluation tasks. We compared the similarity judgments collected from Evalutron6000 (E6K) and MTurk using the Music Information Retrieval Evaluation eXchange 2...
متن کاملUsing Crowdsourcing to Generate an Evaluation Dataset for Name Matching Technologies
Crowdsourcing can be a fast and cost-effective approach to obtaining data for training and evaluating machine learning algorithms. Name matching is the challenging task of identifying which names refer to the same person, which is crucial for effective entity disambiguation and search. While there are a number of name matching technologies available, standardized datasets for evaluating them ar...
متن کاملConducting Online Behavioral Research Using Crowdsourcing Services in Japan
Recent research on human behavior has often collected empirical data from the online labor market, through a process known as crowdsourcing. As well as the United States and the major European countries, there are several crowdsourcing services in Japan. For research purpose, Amazon's Mechanical Turk (MTurk) is the widely used platform among those services. Previous validation studies have show...
متن کاملA Comparative Study of Collaborative vs. Traditional Musical Mood Annotation
Organizing music by emotional association is a natural process for humans, but the ambiguous nature of emotion makes it a difficult task for machines. Automatic systems for music emotion recognition rely on ground truth data collected from humans, and more effective methods for collecting such data are being continuously developed. In previous work, we developed MoodSwings, an online collaborat...
متن کامل